Local Average Treatment Effect Research Articles

Interpretable (not just posthoc-explainable) medical claims modeling for discharge placement to reduce preventable all-cause readmissions or death

We developed an inherently interpretable multilevel Bayesian framework for representing variation in regression coefficients that mimics the piecewise linearity of ReLU-activated deep neural networks. We used the framework to formulate a survival model for using medical claims to predict hospital readmission and death that focuses on discharge placement, adjusting for confounding in estimating causal local average treatment effects. We trained the model on a 5% sample of Medicare beneficiaries from 2008 and 2011, based on their 2009–2011 inpatient episodes (approximately 1.2 million), and then tested the model on 2012 episodes (approximately 400 thousand). The model scored an out-of-sample AUROC of approximately 0.75 on predicting all-cause readmissions—defined using official Centers for Medicare and Medicaid Services (CMS) methodology—or death within 30-days of discharge, being competitive against XGBoost and a Bayesian deep neural network, demonstrating that one need-not sacrifice interpretability for accuracy. Crucially, as a regression model, it provides what blackboxes cannot—its exact gold-standard global interpretation, explicitly defining how the model performs its internal “reasoning” for mapping the input data features to predictions. In doing so, we identify relative risk factors and quantify the effect of discharge placement. We also show that the posthoc explainer SHAP provides explanations that are inconsistent with the ground truth model reasoning that our model readily admits.

Abadie’s Kappa and Weighting Estimators of the Local Average Treatment Effect

Recent research has demonstrated the importance of flexibly controlling for covariates in instrumental variables estimation. In this article we study the finite sample and asymptotic properties of various weighting estimators of the local average treatment effect (LATE), motivated by Abadie’s kappa theorem and offering the requisite flexibility relative to standard practice. We argue that two of the estimators under consideration, which are weight normalized, are generally preferable. Several other estimators, which are unnormalized, do not satisfy the properties of scale invariance with respect to the natural logarithm and translation invariance, thereby exhibiting sensitivity to the units of measurement when estimating the LATE in logs and the centering of the outcome variable more generally. We also demonstrate that, when noncompliance is one sided, certain weighting estimators have the advantage of being based on a denominator that is strictly greater than zero by construction. This is the case for only one of the two normalized estimators, and we recommend this estimator for wider use. We illustrate our findings with a simulation study and three empirical applications, which clearly document the sensitivity of unnormalized estimators to how the outcome variable is coded. We implement the proposed estimators in the Stata package kappalate.

Multiple Discoveries in Causal Inference: LATE for the Party

Multiple discoveries in statistics may be more common than many readers realize. We tell a story of multiple discoveries in causal inference related to a quantity called the local average treatment effect (LATE). The development of LATE by Imbens and Angrist contributed greatly to their winning the 2021 Nobel Prize in Economics. We discuss how multiple investigators were either early or late for the “party” of intermediate and final discoveries involving LATE.

The interpretation of 2SLS with a continuous instrument: A weighted LATE representation

This note introduces a novel weighted local average treatment effect representation for the two-stages least-squares (2SLS) estimand in the continuous instrument with binary treatment case. Under standard conditions, we obtain weights that are nonnegative, integrate to unity, and assign larger values to instrument support points that deviate from their average. Our representation does not require instruments to be discretized nor relies on limiting arguments, such as those used in the definition of the marginal treatment effect (MTE). The pattern of the weights also has a clear interpretation. We believe these features of the representation to be useful for applied researchers when communicating their results. As a direct byproduct of our approach, we also obtain a representation of the 2SLS estimand as a weighted average of treatment effects among “marginal compliance” groups, without having to resort to the threshold-crossing representation underlying the MTE construction. The latter representation has an intuitive interpretation as well.

Defining and Identifying Local Average Treatment Effects.

Defining and Identifying Local Average Treatment Effects.

Improving Estimation Efficiency via Regression-Adjustment in Covariate-Adaptive Randomizations with Imperfect Compliance

Abstract We investigate how to improve efficiency using regression adjustments with covariates in covariate-adaptive randomizations (CARs) with imperfect subject compliance. Our regression-adjusted estimators, which are based on the doubly robust moment for local average treatment effects, are consistent and asymptotically normal even with heterogeneous probabilities of assignment and misspecified regression adjustments. We propose an optimal but potentially misspecified linear adjustment and its further improvement via a nonlinear adjustment, both of which lead to more efficient estimators than the one without adjustments. We also provide conditions for nonparametric and regularized adjustments to achieve the semiparametric efficiency bound under CARs.

A computational approach to identification of treatment effects for policy evaluation

For counterfactual policy evaluation, it is important to ensure that treatment parameters are relevant to policies in question. This is especially challenging under unobserved heterogeneity, as is well featured in the definition of the local average treatment effect (LATE). Being intrinsically local, the LATE is known to lack external validity in counterfactual environments. This paper investigates the possibility of extrapolating local treatment effects to different counterfactual settings when instrumental variables can be only binary. We propose a novel framework to systematically calculate sharp nonparametric bounds on various policy-relevant treatment parameters that are defined as weighted averages of the marginal treatment effect (MTE). Our framework is flexible enough to fully incorporate statistical independence (rather than mean independence) of instruments and a large menu of identifying assumptions beyond the shape restrictions on the MTE that have been considered in prior studies. We apply our method to understand the effects of medical insurance policies on the use of medical services.

Preference Externality Estimators: A Comparison of Border Approaches and IVs

This paper compares two estimators—the Border Approach and an Instrumental Variable (IV) estimator—using a unified framework where identifying variation arises from “preference externalities,” following the intuition in Waldfogel (2003) . We highlight two dimensions in favor of the IV approach. First, an econometric model of the data-generating process reveals that the border approach requires a set of identification assumptions that are not easily satisfied in practice: the ignorance of some payoff-relevant information and conflicting spatial correlation assumptions. The IV approach, in contrast, exhibits greater internal validity because it is derived from the model that generates the data. Second, the border approach suffers from representative issues when the true effect sizes are different between border and off-border regions. We use a common political advertising example to evaluate these estimators and suggest ways to evaluate or limit the above concerns, such as excluding localities that are a large share of the policy making region and evaluating spatial correlations of observables. We find the border approach’s representative issue to be substantial when the ignorance assumption is most plausible and observe that spatial correlations do not reflect those needed in the unobservables for consistency of the estimator. The IV, in contrast, does not exhibit concerns related to local average treatment effects. We also derive the specific conditions when the border approach can reduce bias relative to OLS. This paper was accepted by Raphael Thomadsen, marketing. Funding: This work was supported by National Natural Science Foundation of China (NSFC) [72072004, 72131001]. Supplemental Material: The data files are available at https://doi.org/10.1287/mnsc.2023.4977 .

The heterogeneous impact of college education on happiness by gender

AbstractObjectiveThis study examines the heterogeneous impacts of college education on happiness by gender.MethodsTo yield unbiased impact estimates, we take advantage of a natural experiment known as the graduation quota program, which suddenly and massively expanded the opportunities to attend college in the early 1980s in South Korea. Using whether the birth cohorts were exposed to the graduation quota program as an instrumental variable (IV), this study estimates the longer‐term effects of college education on happiness by gender.ResultsThe estimated local average treatment effect by the IV analysis indicates that men who were induced to attend college by the graduation quota program became happier. However, there was no noticeable impact on women's happiness.ConclusionsConsidering that the subjective happiness in our study was measured in their late 30s and 40s, the impact of college on happiness seems to be long‐lasting for men but not for women.

Efficient and Robust Estimation of the Generalized LATE Model

This article studies the estimation of causal parameters in the generalized local average treatment effect (GLATE) model, which expands upon the traditional LATE model to include multivalued treatments. We derive the efficient influence function (EIF) and the semiparametric efficiency bound (SPEB) for two types of causal parameters: the local average structural function (LASF) and the local average structural function for the treated (LASFT). The moment conditions generated by the EIF satisfy two robustness properties: double robustness and Neyman orthogonality. Based on the robust moment conditions, we propose the double/debiased machine learning (DML) estimator for estimating the LASF. The DML estimator is well-suited for high dimensional settings. We also propose null-restricted inference methods that are robust against weak identification issues. As an empirical application of these methods, we examine the potential health outcome across different types of health insurance plans using data from the Oregon Health Insurance Experiment.

Computer Assisted Learning and Academic Performance in Rural Taiwan

The objective of the current study is to examine the impact of an in-school computer-assisted learning (CAL) intervention on the math achievement of rural students in Taiwan, including a marginalized subgroup of rural students called Xinzhumin, and the factors associated with this impact. In order to achieve this, we conducted a cluster randomized controlled trial involving 1,840 fourth- and fifth-grade students at 95 schools in four relatively poor counties and municipalities of Taiwan during the spring semester of 2019. While the Intention-To-Treat (ITT) analysis found that the CAL intervention had no significant impacts on student math achievement, the Local Average Treatment Effect (LATE) analysis revealed significant associations with the math performance of the most active 20% of students in the treatment group. LATE estimates suggest that using CAL for more than 20 minutes per week for ten weeks corresponds to higher math test scores, both in general (0.16 SD–0.22 SD), and for Xinzhumin students specifically (0.3 SD–0.34 SD). Teacher-level characteristics were associated with compliance rates.

The Finite Sample Performance of Instrumental Variable-Based Estimators of the Local Average Treatment Effect When Controlling for Covariates

AbstractThis paper investigates the finite sample performance of a range of parametric, semi-parametric, and non-parametric instrumental variable estimators when controlling for a fixed set of covariates to evaluate the local average treatment effect. Our simulation designs are based on empirical labor market data from the US and vary in several dimensions, including effect heterogeneity, instrument selectivity, instrument strength, outcome distribution, and sample size. Among the estimators and simulations considered, non-parametric estimation based on the random forest (a machine learner controlling for covariates in a data-driven way) performs competitive in terms of the average coverage rates of the (bootstrap-based) 95% confidence intervals, while also being relatively precise. Non-parametric kernel regression as well as certain versions of semi-parametric radius matching on the propensity score, pair matching on the covariates, and inverse probability weighting also have a decent coverage, but are less precise than the random forest-based method. In terms of the average root mean squared error of LATE estimation, kernel regression performs best, closely followed by the random forest method, which has the lowest average absolute bias.

The effects of land direct payment program on farm income and production practices

AbstractMany countries of the world have implemented subsidy programs to maintain a minimum amount of farm production especially for rice. Agricultural support programs usually cause significant fiscal burdens on government budgets. These programs can be generally categorized into decoupled and coupled programs, and one longstanding research interest of previous literature is whether these programs are decoupled or coupled with farm production practices. In Taiwan, a unique land direct payment (LDP) program to rice farms was implemented to ease severe financial burden of the price support program with government purchase. In contrast to production control programs imposing quantity restrictions on farm production, this program is voluntary for rice farms, and participants are paid at a fixed rate per hectare of farmland. This paper examines the impacts of this program on farm income and farm production practices in Taiwan. The causal effect of the program is identified based on the exogenous rollout of the policy across multiple times and areas. We construct a dataset linking nationally representative rice farm surveys over several years and administrative records on program recipients to estimate the local average treatment effect of the program. We find that the effect is 4% of farm income. In addition, program participants rely less on price support programs and produce less rice and use less fertilizer, pesticide, and machinery. Although the LDP reduces rice production, we find that it causes overall government spending on the price support and direct payment programs to increase by 8.7% after the LDP.

Bounding program benefits when participation is misreported

Instrumental variables (IV) are commonly used to estimate treatment effects in case of noncompliance. However, program participation is often misreported in survey data and standard techniques are not sufficient to point identify and consistently estimate the effects of interest. In this paper, we show that the identifiable IV estimand that ignores treatment misclassification is a weighted average of local average treatment effects with weights that can also be negative. This is troublesome because it may fail to deliver a correct causal interpretation, and this is true even if all the weights are non-negative. Therefore, we provide three IV strategies to bound the program benefits when both noncompliance and misreporting are present. We demonstrate the gain of identification power achieved by leveraging multiple exogenous variations when discrete or multiple-discrete IVs are available. At last, we use our new Stata command, ivbounds, to study the benefits of participating in the 401(k) pension plan on savings.

Double robustness for complier parameters and a semi-parametric test for complier characteristics

Summary We propose a semi-parametric test to evaluate (a) whether different instruments induce subpopulations of compliers with the same observable characteristics, on average; and (b) whether compliers have observable characteristics that are the same as the full population, treated subpopulation, or untreated subpopulation, on average. The test is a flexible robustness check for the external validity of instruments. To justify the test, we characterise the doubly robust moment for Abadie’s class of complier parameters, and we analyse a machine learning update to weighting that we call the automatic $\kappa$ weight. We use the test to reinterpret Angrist and Evans' different local average treatment effect estimates obtained using different instrumental variables.

New evidence on the relationship between mental and physical health

Past research has documented that mental health has a causal impact on physical health. This represents a local average treatment effect, but policymakers are primarily interested in mental health changes for the entire population, not just those exposed to an arbitrary exogenous shock. The need for reliable quantitative evidence presents a notable empirical challenge. To fill this research void, we use a recent method proposed by Oster (2019) to assess the reliability of the instrumental-variable estimates. Our findings support the soundness of the instrumental-variable estimates in the present context, meaning that researchers can have a good degree of confidence in using such estimates for policy formulation.

Student Success Coaching Is Best Delivered Early and Often

This research investigates the effects of a success coaching program by examining the timing of the first appointment and the frequency of additional coaching. Using a fuzzy regression discontinuity approach, we provide Local Average Treatment Effects of the program when students make appointments early in the semester. Students with appointments scheduled before discontinuity had an average of two additional meetings compared to students who waited to make their first appointment. We find each coaching session yields an effect of about 0.2–0.3 grade points and approximately 0.8 additional credits earned. This article contributes to understanding of the timing and frequency effects for success coaching and recommends programs aim to have first interactions with students happen before the sixth week of classes and offer four to six appointments per semester for maximum impact.

Inference under covariate-adaptive randomization with imperfect compliance

This paper studies inference in a randomized controlled trial (RCT) with covariate-adaptive randomization (CAR) and imperfect compliance of a binary treatment. In this context, we study inference on the local average treatment effect (LATE), i.e., the average treatment effect conditional on individuals that always comply with the assigned treatment. As in Bugni et al. (2018, 2019), CAR refers to randomization schemes that first stratify according to baseline covariates and then assign treatment status so as to achieve “balance” within each stratum. In contrast to these papers, however, we allow participants of the RCT to endogenously decide to comply or not with the assigned treatment status.We study the properties of an estimator of the LATE derived from a “fully saturated” instrumental variable (IV) linear regression, i.e., a linear regression of the outcome on all indicators for all strata and their interaction with the treatment decision, with the latter instrumented with the treatment assignment. We show that the proposed LATE estimator is asymptotically normal, and we characterize its asymptotic variance in terms of primitives of the problem. We provide consistent estimators of the standard errors and asymptotically exact hypothesis tests. In the special case when the target proportion of units assigned to each treatment does not vary across strata, we can also consider two other estimators of the LATE, including the one based on the “strata fixed effects” IV linear regression, i.e., a linear regression of the outcome on indicators for all strata and the treatment decision, with the latter instrumented with the treatment assignment.Our characterization of the asymptotic variance of the LATE estimators in terms of the primitives of the problem allows us to understand the influence of the parameters of the RCT. We use this to propose strategies to minimize their asymptotic variance in a hypothetical RCT based on data from a pilot study. We illustrate the practical relevance of these results using a simulation study and an empirical application based on Dupas et al. (2018).

Does higher education affect pro-environmental behavior? Evidence from household waste recycling in Greece

We examine the effect of higher education attainment on pro-environmental behavior focusing on household waste recycling. To address the endogeneity of higher education attainment, we exploit a set of reforms that increased opportunities for university studies in Greece, affecting cohorts graduating from high school in year 2000. We leverage the exogenous variation introduced by the school enrollment age cutoff and estimate the local average treatment effect of higher education employing a fuzzy regression discontinuity design, comparing educational attainment and recycling behavior between households that were just, and just not exposed to the reforms. We find little evidence that higher education increases the probability of recycling, and no evidence that the share of household waste recycled responds to higher educational attainment. Our results suggest that human capital accumulation alone may not deliver green behavior returns.

It is never too LATE: a new look at local average treatment effects with or without defiers

Summary In heterogeneous treatment effect models with endogeneity, identification of the local average treatment effect (LATE) typically relies on the availability of an exogenous instrument monotonically related to treatment participation. First, we demonstrate that a strictly weaker local monotonicity condition—invoked for specific potential outcome values rather than globally—identifies the LATEs on compliers and defiers. Second, we show that our identification results apply to subsets of compliers and defiers when imposing an even weaker local compliers-defiers assumption that allows for both types at any potential outcome value. We propose estimators that are potentially more efficient than two-stage least squares (2SLS) in finite samples, even in cases where 2SLS is consistent. Finally, we provide an empirical application to estimating returns to education using the quarter of birth instrument.